As a method of executing recognition or position/orientation measurement processing from an image of a measurement object included in a grayscale image or range image, Model-Based Vision is known. The range image is an image in which each pixel has depth information. This method calculates correspondences between geometric features detected from a captured image and those extracted from a three-dimensional shape model of a measurement object, and estimates the recognition or position/orientation measurement parameters to have distances between the correspondences as an evaluation function. This technique requires the three-dimensional shape model of the measurement object beforehand. A method which diverts design data of CAD software as this three-dimensional shape model is called CAD-Based Vision.
In general, CAD data which expresses the shape of an object is normally expressed by a parametric shape model. The parametric shape model expresses a shape by combining a plurality of parametric curved surfaces such as B-Spline curved surfaces or NURBS (Non-Uniform Rational B-Spline) curved surfaces. Since the parametric curved surface defines a curved surface by a combination of a basis function and parameters, it can express a complicated curved surface shape by a small information amount as its advantage. Therefore, when the parametric shape model, which compactly holds shape information, is used at the time of design in the recognition or position/orientation measurement processing of an object, geometric features on the model can be calculated with high accuracy, thus attaining accurate processing.
For example, P. J. Besl and N. D. McKay, “A method for registration of 3-D shapes,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239-256, 1992 discloses an Iterative Closest Point (ICP) algorithm as a method of estimating the position and orientation of a target object from a captured range image using a parametric shape model. More specifically, the following method is disclosed. That is, the position and orientation of an object are estimated by fitting a three-dimensional shape model of the object into three-dimensional point cloud data converted from a range image. Based on approximate values of the position and orientation, geometric features of the shape model, which are closest to respective three-dimensional points, are searched, and update processing of the position and orientation is repeated to minimize the sum total of distances between the points and geometric features of the shape model.
In general, since the parametric shape model requires complicated geometric operations and product-sum operations using a plurality of parameters, a long time period is required for calculations carried out when geometric features on curved surfaces are to be sequentially searched. For this reason, a long processing time is required when correspondences between the parametric shape model and captured image are calculated. For example, in the aforementioned registration method using the parametric shape model (see P. J. Besl and N. D. McKay, “A method for registration of 3-D shapes,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239-256, 1992), correspondences between the three-dimensional point cloud converted from the range image and geometric features on the parametric shape model are calculated. In this case, the geometric features on the parametric curved surfaces have to be sequentially searched. Upon sequentially searching the geometric features, a long processing time is required to calculate the coordinates and normals of points on the parametric curved surfaces. Therefore, such long processing time adversely influences a response time in real-time processing that executes three-dimensional measurements using correspondences with a captured image.
Also, U.S. Pat. No. 6,504,957 discloses a method to extract regions where geometric features are easily detected in a captured image from a shape model in advance. With this method, as regions where measurement tends to be stable on the captured image, those having small curvatures on the shape model are extracted as geometric features. However, this method does not consider the position and orientation of an image capturing device with respect to a measurement object. Geometric features such as edges and planes which are associated with each other between the shape model and captured image largely change depending on the observation direction of the measurement object, that is, the position and orientation of the image capturing device with respect to the measurement object. For this reason, the method described in U.S. Pat. No. 6,504,957 impairs the accuracy of extracted geometric features when the image capturing device is located at a slant position with respect to the measurement object.
Furthermore, Japanese Patent Laid-Open No. 2002-178576 discloses a method to speed up calculations of geometric features on a model by parameter-converting a parametric shape model into simple equivalent curved surfaces. However, this method aims to render the parametric shape model at high speed, and it consequently requires sequential search processing so as to calculate correspondences with measurement points.
In consideration of the aforementioned problems, the present invention provides a technique for efficiently calculating geometric features required to use correspondences with a captured image based on an observation direction of an object in recognition or position/orientation measurement processing of the object using a parametric shape model.